Machine learning and the “magic wand” problem

I’ve never been a fan of “magic wand” questions during user interviews:

“If you could wave a magic wand and solve the problem, how would you do it?”

Co-creation is important! But I don’t think this kind of question is helpful, especially when it comes to machine learning or “artificial intelligence” – however we define that.

Part of the problem is that people have a hard time accurately predicting what they’ll do in a hypothetical future. This is well-understood in UX, and part of the reason why we don’t ask “how likely are you to use this thing which doesn’t exist yet?”

But the biggest issue, especially when it comes to machine learning, is that a lot of people have been led to believe that there are no constraints to what “artificial intelligence” can do. It’s not their fault – in fact, it’s mostly ours, since most of tech has been pushing AI Hype as far as we can get away with.

As a result, there’s a risk that people will suggest ideas that are impossible and/or unethical. If interview participants say “I want a way to automatically analyze resumes and select the best candidates,” or “I want to be successful in my classes without doing any studying,” we can approach those as a starting point for further conversation, but we’d never want to actually implement those.

I’m not saying “don’t listen to users.” We should absolutely listen to users! In fact, we should be designing with communities instead of for them. We absolutely need to cut it out with the idea that “users don’t know what they want,” along with the overplayed “faster horse” quote. Laura Klein and Kate Rutter did a fabulous episode of What’s Wrong With UX about that.

So what should we do?

I think that co-creation continues to be the key. We can approach communities in a genuine spirit of partnership. That means developing a deep understanding of the communities we’re working with, and being open about the constraints and limitations of our technology, as well as our philosophical and ethical values. In all likelihood, surveys and focus groups won’t be enough.

So instead of brandishing a magic wand, some questions we might ask instead are:
• How are people solving this problem now? How is that working for them?
• Who wants the status quo to change? Who benefits from it staying the way it is now?
• What are the systems and power dynamics involved?
• If we fixed the problem in a particular way, what would be the secondary effects? Who would benefit, and who would be harmed? What new problems would arise?